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Small-scale farm forestry has the potential to offer many benefits both to 
landholders and the wider community. As with all changes in land-use 
practices, there are associated benefits and costs and these are not uniformly 
distributed. They have varying impacts on the different values, aspirations, 
goals and objectives that exist within the community. Furthermore, the 
community does not consider these values, aspirations, goals and objectives 
of equal importance. The degree of concern can vary from minor to high and 
overriding all other considerations. When evaluating farm forestry options it 
is necessary to address all of these concerns. This paper examines the 
combined use compensatory and non-compensatory multi-criteria analyses to 
evaluate forestry options, in a case study for the Darling Downs region of 
Queensland, Australia. These aggregation techniques are found to be highly 
complementary and together provide a comprehensive analysis. The 
compensatory technique provides a sound measure of overall performance of 
a forestry system, whereas the non-compensatory technique alerts decision¬ 
makers to presence of particularly poor performance with respect to 
individual criteria. The compensatory technique used is simple and 
understandable even for those with non-mathematical backgrounds. This 
analysis can identify and aid communication of the relative benefits and costs, 
and trade-offs, between economic, environmental and social considerations. 

Keywords: Multi-Objective Decision Support Systems, farm forestry 
options, trade-offs, community participation 


INTRODUCTION 

Small-scale farm forestry could form an integral part of the rural landscape in many 
areas of Australia, providing economic, environmental and social benefits. Farm 
forestry may help to attain environmental quality objectives, such as reduction of 
demand to clear remnant vegetation, soil retention, improved water quality, carbon 
sequestration and salinity mitigation. From an economic perspective, forestry may 
diversify the products of a farming community and reduce risk associated with 
reliance on a single crop, provide a long-term and continuing product and revenue 
stream, or provide a nest-egg to be harvested at retirement. The extent to which 


99 



100 


D.I. Jeffreys 


various farm forestry practices fulfil the objectives of economic enhancement, 
ecological enhancement and social and community enhancement needs to be 
assessed before appropriate systems can be recommended. In addition, a proposed 
change in practice should not seriously disadvantage any group in the local 
community or undermine the objectives of these groups. 

Multi-criteria analysis is a decision-support process that aids decision-makers by 
providing a framework to gather and display the required data in a clear and 
transparent manner (Froger and Munda 1998). A multi-criteria analysis includes a 
finite number of alternative plans or options, a set of criteria by which the 
alternatives are to be judged, and a method for ranking the options based on how 
well they satisfy the criteria (RAC 1992). This process includes weighting and 
evaluating criteria, and methods to aggregate data when required (Froger and Munda 
1998). 

This paper evaluates the utility of using compensatory and non-compensatory 
multi-criteria analysis techniques to evaluate and compare various options for farm 
forestry development on the Darling Downs in southern Queensland, Australia. The 
use of multi-criteria analysis for evaluating farm forestry or other land-use practices 
in Australia has focused on evaluating alternatives from an economic, ecological, 
social and cultural perspective, and thus identifying trade-offs between these 
considerations. Further to these trade-offs, fatal flaws of each possible land-use need 
to be identified. This analysis incorporates community and other stakeholder opinion 
combined with expert opinion in a transparent approach consolidating the available 
data in a single framework. 

The paper first briefly reviews compensatory and non-compensatory aggregation 
techniques used in multi-criteria analysis. The utility of applying two of these 
techniques in combination for land-use management is then examined. For the case 
study, the process of eliciting stakeholder driven options and expert evaluations of 
the relative performance of the options is described. The farm forestry options and 
evaluation criteria for the case-study area are then outlined. Finally, the outcomes 
and conclusion of the case study and an appraisal of the process and analysis are 
presented. 


COMPENSATORY AND NON-COMPENSATORY MULTI-CRITERIA 
ANALYSES 

In multi-criteria analysis, relative weights are placed on the decision criteria by 
which the options are evaluated. This recognises that some of the criteria are more 
important to the stakeholders than others, and should attract greater weights in the 
analysis. Second, the scores that the options receive against each criterion judging 
their performance against that criterion (which form the effects table) are 
aggregated, i.e. combined to create an overall score for each option. A wide variety 
of techniques are available, these can be split into compensatory and non¬ 
compensatory techniques. In compensatory techniques poor performance in a 
number of criteria can be compensated for by high performance in the other criteria 
and may not be reflected in the aggregated performance of an option. In non¬ 
compensatory techniques poor performance in a number of criteria cannot be 
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compensated for by high performance in the other criteria and will be reflected in 
the aggregated performance of an option. 

The more commonly used techniques are now reviewed and their strengths and 
weaknesses discussed. 

The Multi-Attribute Utility Theory (MAUT) of Weighting and Aggregating 

MAUT provides the theoretical basis for multiple criteria methods. MAUT states 
that the overall utility of an option (an option’s attractiveness to the decision-maker) 
can be derived from the sum of the measured values of its attributes. The attributes 
are transformed into single dimensionless utilities using utility probability functions. 
These probability functions are assessed for each attribute and are often complex. 
They are the basis for standardising functions used in many evaluation methods 
(Keeney and Raffia 1976). 

The SMARTER Method of Weighting and Aggregation 

The SMARTER technique (Edwards and Barron 1994) has three variations, namely 
SMART, SMARTS and SMARTER, referring to Simple Multi-attribute Rating 
Technique, Simple Multi-attribute Rating Technique using Swings, and Simple 
Multi-attribute Rating Technique Exploiting Ranks. Edward and Barron asserted 
that SMART contained an error in logic, which was subsequently corrected in 
SMARTS. These techniques use linear approximation to standardise the scores to 
single dimensionless cardinal utility functions and a weighted additive aggregation 
function. 

The Analytical Hierarchy Process (AHP) for Weighting and Aggregation 

The AHP (Saaty 1980) provides a hierarchic framework for the evaluation of 
decision problems using multiple criteria. The AHP uses the definitions of relative 
importance to evaluate the weights placed on individual decision criteria and the 
score received by each option for a given decision criterion. The criteria are 
evaluated using pairwise comparisons; each criterion is assessed individually against 
all the other criteria. These comparisons are made by the stakeholders (or decision¬ 
maker) and are in the form of a statement of the relative importance of the pair of 
criteria. A clear advantage of this method is the ease with which stakeholders can 
answer the pairwise question, but this assumes that decisions are made under 
complete knowledge about the issue (Janssen 1991). Inconsistency is a major issue 
with AHP, in which circular statements can arise, e.g. Criterion 1 is preferable to 
Criterion 2, Criterion 2 is preferable to Criterion 3, and Criterion 3 is preferable to 
Criterion 1. To counter this, an inconsistency ratio is calculated. Saaty (1994) 
asserted that an inconsistency of less than 0.1 (on a scale up to 5.84) represents a 
tolerable error in measurement and does not bias the result. With a small number of 
criteria it is possible to achieve consistency according to this measure, but as the 
number of criteria increases achieving consistency becomes increasingly difficult 
(Robins 1999). To address this problem Saaty (1980) suggested a maximum of 
between five and nine criteria. 
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Rank Order Methods of Weighting 

Rank order weighting techniques are commonly used for assigning relative weights 
to a set of criteria. These weighting techniques assign weights to individual criteria 
according to a statement of importance or rank. These are usually in the form 
‘Criterion 1 is more important than Criterion 2, which is more important than 
Criterion 3, which is more important Criterion 4’, and so on. Given these statements, 
a set of feasible weights can be defined, as well as the probability density function 
for all the feasible weights. The Rank Order Centroid (ROC) - as used in the 
SMARTER technique (Edwards and Barron 1994) - and the Expected Value 
Method (Nijkamp et al. 1990), use the centroid value as the weight for the criteria. 
The centroid value is also the most probable or expected value. Alternatively, the 
Range of Values method (Yakowitz et al. 1992) reports the extreme values of this 
probability density function, the maximum additive value and the minimum additive 
value. 

The Weighed Summation Method of Aggregation 

Weighted summation (Janssen 1991; Yakowitz et al. 1997) is the simplest and most 
commonly used aggregation technique. It is also described as the multi-attribute 
value (MAV) technique (Barron and Barrett 1996). The value accorded to an option 
is the sum of the value the option receives for each criterion multiplied by the weight 
given to that criterion. The weights for the criteria are assigned by the decision¬ 
maker or stakeholders to reflect the relative importance they place on the criteria. 
Weights may be directly assigned or assigned using a ranking process or pairwise 
comparisons that are usually embedded in the tool being used. The weighted 
summation technique is one of many methods included in Multi-Attribute Utility 
Theory (see Keeney and Raffia 1976), and whilst other techniques have a stronger 
theoretical basis they are rarely used because they are complicated and time 
consuming (Winterfeldt and Edwards 1986). 

The Electre Method of Aggregation 

The Electre (E/imination et choix fraduisant la realite or Elimination and choice 
corresponding to reality) method (Cook et al. 1988, Roy 1991, Janssen 1991, p. 61) 
uses pairwise comparison of options with regards to the measured value of each 
criterion. Each pairwise comparison assesses the criteria using a concordance and a 
discordance measure. The concordance index is derived from the differences 
between all criteria for the two options. The discordance index reflects the maximum 
difference between any individual criterion in the two options. Threshold values are 
set for concordance and discordance. For example, when comparing option 1 with 
option 2 for a given criterion, option 1 is said outperform option 2 if the following 
statement is true: 

The difference between the criterion value for option 1 compared to that for 
option 2 is greater than the concordance threshold and smaller than the 
discordance threshold. 

The concordance index measures the relative overall performance of the options, 
while the discordance index measures poor performance in individual criteria. The 
rationale of the discordance index is that overall good performance in most criteria 
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cannot compensate for poor performance in an individual criterion. Five versions of 
the Electee method have been proposed (Roy 1991; Simpson 1996), these being 
Electee I, II, III, TV and TRI. Electee I can be used for selection problems, Electee 
TRI for assignment problems, and Electee II, III and IV for ranking problems. 
Electee II is used in this study. A detailed description of the Electee II method can be 
found in Jeffreys and Lawrence (in press). 

The Regime Method of Aggregation 

The Regime method (Hinloopen and Nijkamp 1990) uses pairwise comparison of 
options to determine dominance. Each option is compared on a criterion-by-criterion 
basis, for a given criterion against the same criterion in a second option. If a given 
option outperforms another option with respect to every criterion, the given option is 
said to strictly dominate the other option. However, usually the number of criteria 
against which performance is highest varies between options. Under the Regime 
method, the probability that an option is dominant is calculated (as the number of 
dominant criteria over the total number of criteria). The sum of these pairwise 
probabilities of dominance is the overall probability of dominance, and this is the 
measure by which the options are ranked. 

The Evamix Method of Aggregation 

Evamix (Voogd 1983) also determines dominance of an option on a criterion-by¬ 
criterion basis; it is especially designed for mixed (quantitative and qualitative) data. 
The quantitative and qualitative scores are standardised separately to a 0 to 1 range. 
The degree of pairwise dominance for each pair of criteria is calculated, as the 
difference in score received by the higher performing option compared to the poorer 
performing option. The value assigned to each option is the weighted sum of the 
dominance scores. The weights can be derived by any of the weighting techniques. 
The outcome of this aggregation is a similar to the outcome of the weighted sum; the 
relative performance of the options is the same but there is difference in the scale of 
the measure of performance. 

In each of the Evamix, SMARTER, MAUT, AHP and the weighted summation 
techniques, the performances of the criteria are aggregated using a weighted sum. 
The Regime technique considers only the number dominant criteria disregarding the 
degree of dominance. In this respect all of these techniques are compensatory 
techniques, in that poor performance in any one criterion can be compensated by 
overall good performance in the other criteria and that one criterion can be traded off 
against another (Cook et al, 1988, p. 901). Evamix, SMARTER, MAUT, AHP, and 
the weighted summation techniques differ little in their outcomes, given the same 
effects table and weights are used. SMARTER, MAUT and AHP employ a weighted 
additive function to aggregate the scores received by each option. The techniques 
vary in the methods employed to transform measured data into the single 
dimensionless utilities used when aggregating the option’s performance against the 
individual criteria. AHP uses expert opinion of the relative performance of the 
options against each criterion in turn to quantify the single dimensionless utilities. 
SMARTER employs linear approximations to standardise the measured data for 
each criteria and quantify the single dimensionless utilities and MAUT employs 
utility probability functions. Linear approximations are not valid in all cases and 
utility probability functions tend to be complex, difficult to evaluate and highly 
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specific to a given context. The weighted summation technique (Janssen 1991) uses 
linear and non-linear approximations as appropriate to quantify the single 
dimensionless utilities. This technique is simple and most likely to gain acceptance 
from the stakeholders. 

Electre is the only widely used non-compensatory technique. Non-compensatory 
techniques offer benefits to decision-makers not provided by the compensatory 
techniques, these will be discussed in the next section. However, the algorithms are 
complex and this technique may have problems gaining acceptance from the 
stakeholders. 


THE USE OF COMPENSATORY AND NON-COMPENSATORY 
TECHNIQUES IN COMBINATION 

As a result of the above review of techniques, this study focuses on use of two 
aggregating techniques in combination with a single weighting technique. These are 
the weighted summation (Janssen 1991; Yakowitz et al. 1997) and Electre II 
aggregation techniques (Roy 1991), and the Rank Order Centroid (ROC) weighting 
technique (Edwards and Barron 1994). The ROC technique was chosen for this 
study because it is easy to elicit the rank orders from the stakeholders and this 
technique has a strong theoretical basis (Edwards and Barron 1994). 

The weighted summation technique provides a sound measure of the overall 
performance, is logically simple and easily gains the acceptance of stakeholders. It is 
a compensatory aggregation method. Compensatory methods provide an overall 
measure of performance but tend towards the mean; that is, the result of using a 
weighted average tends to produce results that cluster around the mid-point of the 
score range. High performance in the majority of criteria will compensate for poor 
performance in one or more criteria. An option that has high performance in all 
criteria may have the same aggregate performance as an option that has 
exceptionally high performance in a large number of criteria and exceptionally poor 
in some criteria. 

Electre II is a non-compensatory aggregation method which checks for poor 
performance in a number of criteria, by using the discordance test. Poor performance 
in one or more criteria amongst high performance in the other criteria will be 
reflected in poor aggregate performance. An option with sound to high performance 
in all criteria will outperform the option with exceptionally high performance against 
the majority of criteria and exceptionally poor performance against a few criteria. 
Electre II is a complicated method and is not easily communicated to, or understood 
by, persons without an advanced understanding of mathematics. Consequently, the 
method has not received wide application to natural resource management outside of 
theoretical studies. 

When there is general agreement about ranking of options using the two 
aggregation techniques, no further investigation is required. If there is disagreement 
between the techniques and the performance of one or more options is relatively low 
using the Electre analysis compared to weighted summation, it can be concluded that 
these options have one or more poor performing criteria. These criteria can be 
identified by examining the effects table. The poor performance according to these 
criteria should then be judged as to importance in land-use decisions. If the criteria 
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have been identified as highly important to the stakeholders then the options should 
be considered as undesirable options. If the criteria have been identified as having 
low importance to the stakeholders the options may still be considered as desirable. 

As regards computer software, the package Definite (Janssen 1991 and Janssen et 
al. 2001) is a high-end multi-criteria analysis tool, modular in structure and with 
high functionality. The problem definition module includes the ability to construct 
criteria hierarchies and the effects table, as well as evaluate the options by using 
various methods, and can undertake correlation analyses on the criteria. The multi¬ 
criteria analysis module includes many aggregation and weighting techniques. 
Definite was chosen for this application because it includes all the multi-criteria 
analysis discussed, including the Electre and weighted summation aggregation 
techniques and the Rank Order Centroid weighting technique. Definite also includes 
functionality for cost-benefit analyses and sensitivity analyses. 

Mathematics of the ROC, Weighted Summation and Electre II Techniques 

The Rank Order Centroid Weighting Technique 

This weighting technique converts the rank order of criteria importance into a set of 
ordinal weights following a simple algorithm described by Edwards and Barron 
(1994). Given that criterion 1 in more important than criterion 2, and criterion 2 is 
more important than criterion 3 (Cri^Cr^C^), the weight assigned to each criterion 
would follow a similar rule wi>w 2 >W 3 , where Wj is the possible weight applied to 
criterion j. 

If Crj > Cr 2 > . .. > Cq 

then wi > w 2 > . . . > Wj 

If k is the number of criteria 

then wi = (1+1/2+1/3+ . . . +l/k)/k 

w 2 = (0+1/2+1/3+ . . . +l/k)/k 
w 3 = (0+0+1/3+ .. . +l/k)/k 

wj = (0+ . . . +0+l/k)/k 

More generally, the weight of criterion j is: 

k 

Wj=(l/k) I (1/i) 

H 


Equation 1 
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The Weighted Summation Aggregation Technique 

The weighted summation aggregation technique takes the form (Janssen 1991, 
Barron and Barrett 1996): 


V(w,v) = Lj Wj Vj Equation 2 

where V = the weighted value or overall score for a given option 

Wj = the weight for a given criterion j 

Vj = the value or score for a given option with respect to criterion j 
The Electre II Aggregation Technique 

The first step in the Electre II technique, as described by Goicoechea et al. (1982), 
Roy (1991) and Simpson (1996), is to establish a complete rank order of the options 
using a number of tests. The first three of these are the indifference test, the 
preference test and the veto test. These are evaluated between two options for each 
of the criteria. These tests are now outlined. 

1. The indifference test is designed for testing that Option A is indifferent to 
another Option B (notation AIB ) for a given criterion. This allows for 
options with a small variation in score to be considered equal. In Electre II 
the decision-maker sets the indifference measure q for each criteria. 

AIB if | aj - bj | < qj Equation 3 

where aj is the performance (the score it receives in the effects table) of 
option A on criterion j, bj is the performance of Option B on criterion j, and 
qj the indifference measure. The indifference measure will vary depending 
on the score range of the criteria. 

2. The preference test evaluates whether an Option A is preferred to another 
Option B (notation APE), that is Option A outperforms or outranks Option 
B. This determines a strong or strict preference for Option A; that is, the 
score received by option A for criterion j is considerably better than that for 
option B. The preference threshold pj is the level at which the decision¬ 
maker considers Option A to strongly outperform Option B. 

APB if aj > bj + Pj Equation 4 

where «,• is the performance of option A on criterion j, bj is the performance 
of option B on criterion j, and pj is the strong preference threshold. This test 
is applied to all the criteria and the results from all the criteria are 
considered further in the analysis. 

3. The veto test evaluates whether an Option B is not preferred to another 
Option A; that is, the score received by option B for criterion j is 
considerably worse than that for option A (notation BPA ). The Veto level Vj 
is the level at which Option B is not preferred to Option A, for any criterion. 
If the option B performs at a level that is lower than the performance of 
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option A by a greater than the veto level for any criterion, Option B is 
always not preferred to Option A, regardless of the scores in the other 
criteria. 


BPA if aj + Vj < bj Equation 5 

where Uj is the performance of option A on criterion j, bj is the performance 
of option B on criterion j, and y, is the veto measure. This test is applied to 
all the criteria. Failing this test in one criterion will result in Option B 
always being considered not preferred to Option A. 

The performance of a set of two options for each criterion can be assigned. This can 
be categorised into four groups. The first is indifferent / group, the comparisons that 
fulfil equation 3, the second the strongly preferred P group, the comparisons that 
fulfil equation 4 or equation 5. The third group Q is a group of weak preference all 
those that fall between the indifferent set and the strong preference set. A fourth 
group R is created from the residual - those that failed all of the above tests - and 
these are deemed to be incomparable. 

Using these threshold tests and the subsequent group Electre II technique builds 
an outranking relationship S, for each criterion. This group (notation aSjb) is the set 
for each criterion where option A is at least as good as Option B. 

Therefore for criterion j: 

aSjb:A is at least as good as B on criterion j if a, < b, - qj 

{aSjb} ={aPjb} U {aQjb} U {aljb} Equation 6 

Using the groups S, P, and Q, Electre specifies values known as the concordance and 
discordance coefficients. The concordance coefficient is a measure of the strength of 
the argument A is at least as good as B, c(A,B). The discordance coefficient is the 
strength of the counter argument; that is A is not as good as B, d(A,B). In many 
examples the criteria do not carry equal weights. Weights are applied to each 
criterion using the ROC technique. These weights are included in defining the 
Concordance and Discordance indices. 

c(A,B) = £ P (wj) / £ (wj) 

Using the ROC weighting technique, E (Wj)= 1, so in this example 

c(A,B) = I P (wj) Equation 7 

The next stage of the Electre II technique requires the decision-maker to set 
concordance and discordance thresholds. The level of the concordance threshold c 
and the discordance threshold d are set according to the aims of the analysis. High 
levels are likely to filter out all but the better performing options, while lower levels 
will identify acceptable and unacceptable groups of options. That is, when the 
concordance threshold c and the discordance threshold d are set to a high level, 
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relative to the range of scores received by the options, the most preferred options 
will have substantially higher performance in most criteria and no critically poor 
performing criteria. 


The following decision rules are applied to the pairwise comparison of all the 
options against each other. 


A outranks B if c(A,B) > c 

A outranks B if c(A,B) > c 

c(B,A) > c* 
d(B,A) < d* 

A and B are incomparable if 

c(A,B) > c* and d(A,B) > d* 
c(A,B) > c* and c(B,A) < c* 
c(A,B) < c* and c(B,A) < c* 


and d(A,B) < d* 

and 

and 


or 

or 

and d(B,A) < d* 

Equation 8 


Statements of outranking can be drawn from the outcomes of the decision rules. The 
statements of outranking can then be interpreted as an ordinal ranking of the options. 
Options that are incomparable or outrank the same group of options should be given 
equal ranking. These ranks can be displayed as lists of position or as bar charts. 


CASE STUDY BACKGROUND, METHOD, OPTIONS AND CRITERIA 

A multi-criteria analysis was conducted in which compensatory and non¬ 
compensatory multi-criteria analysis techniques were applied to compare potential 
forestry options for the Hodgson Creek catchment on the Darling Downs in southern 
Queensland. The eastern edge of the Hodgson creek catchment is formed by the 
western slopes of the Great Dividing Range, which descends steeply to the plains 
that cover the majority of the catchment. The high land areas lie on the south-west 
edge of the city of Toowoomba and are predominantly rural residential. The west of 
Hodgson Creek catchment contains fertile cropping land with some grazing country. 
The catchment contains little remnant vegetation, comprising mainly dry open 
sclerophyll woodland in the highland areas and native grasslands in the drainage 
gullies. The catchment is in the medium to low rainfall zone with average annual 
rainfall of approximately 700 mm. 

The purpose of this study was to provide an assessment of the various options for 
land-use change which includes farm forestry as a new land-use in Hodgson creek. 
The assessment was required to identify the economic, environmental and social 
value of alternative farm forestry options. A set of decision criteria that reflected the 
objectives of economic enhancement, environmental enhancement and social and 
cultural enhancement, was developed with input from various stakeholders in 
Hodgson creek. Furthermore, the assessment sought to identify trade-offs between 
these objectives. 
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The Process of Developing a MCA for Farm Forestry in Hodgson Creek 

The MCA for farm forestry in the Hodgson Creek catchment was developed through 
an extended process including consultation with stakeholders and technical experts. 
The stakeholders involved in this study included local landholders (all broadacre 
farmers), the local Landcare coordinator, forestry industry representatives, local 
government officers (Pittsworth Shire Council), and state government extension 
staff in the Department of Natural Resources and Mines (NR&M) and the 
Department of Primary Industries (DPI). The expert group included researchers, 
extension staff and managers from state government agencies (NR&M and DPI) and 
the University of Southern Queensland. Input from stakeholders was obtained in a 
one-day workshop at Felton, within the catchment, in March 2001. Input from 
experts was obtained in two one-day workshops in June 2001. The MCA 
development process proceeded through three phases, as summarised below: 

Phase 1: Development of options, criteria and importance orders 

1. An initial MCA workshop to identify a preliminary list of options and 
criteria was conducted involving only technical experts, without the 
assistance of spatial information. (The results from the initial MCA 
workshop were the starting point of the second analysis, involving both 
stakeholders and technical experts). 

2. The first round results were presented to the stakeholder reference group, 
along with detailed descriptions of the options including maps of the 
possible spatial extent of the options. 

3. Options and criteria were further discussed in the stakeholder workshop. 

4. A final set of options was defined from the preferences of the stakeholder 
groups. Some options were removed or substantially changed to reflect the 
views of this group. 

5. A similar process occurred with the criteria, which were evaluated with 
regards to their relevance and importance considering the values and goals 
of the stakeholder group. During this process, some criteria were added, 
removed or modified. Three stakeholder groups were present at this 
meeting, and each group produced separate importance orders. These 
groups were landholders, DPI and NR&M extension officers, and local 
government officers (of the Pittsworth Shire Council). This new set of 
options, criteria and importance orders formed the basis of the second round 
analysis. 

Phase 2: Development of the effects table 

1. Persons with local farm forestry knowledge were invited to form the 
Technical Reference Group (TRG) for the final analysis. 

2. The options were scored against the criteria, during two focus group 
sessions. The options were assessed using the considered opinions of those 
experts present. 
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3. The options were considered at two time scales, initially defined as the short 
term and long term. The TRG further defined these periods. 1 2 

Phase 3: Analysis of effects table and ranking of options 

1. The effects table was analysed using the MCA tool Definite. The analyses 
used three rankings representing the views of the three stakeholder groups. 
The criteria were also placed into a hierarchy and divided into economic, 
environmental and social criteria. 

2. Two aggregation methods were used, these being the weighed summation 
(chosen for transparency and acceptability with stakeholder groups), and 
Electre II (to highlight any poor performance in individual criteria). 

The outcome of the process for developing the farm forestry MCA was a set of 
farm forestry options considered feasible by both stakeholder and expert groups. A 
set of evaluation criteria reflecting the goals and objectives of the stakeholders - 
including the relative importance of each criterion - was produced. Of the three 
groups present in the stakeholder meeting only the results of the analysis using the 
preferences of the landholders are presented in this paper. In this example, the 
outcomes of the three analyses produced similar results, although this may not be 
true in all situations. Where disagreement appears between stakeholder groups this 
disagreement would be highlighted and discussed by the analyst. It may be the case 
that an option including elements of the most preferred options of the different 
groups could be developed to satisfy all parties. Details of the analyses from all three 
perspectives are provided in Jeffreys (in press). 

The Farm Forestry Options 

In consultation with landholders, government officers, Landcare officers and 
technical experts in farm forestry, eight management practices were defined 
(Jeffreys and Cockfield, in press). 

1. An approximation of current land use 

This scenario is based on assumptions about ‘typical’ land uses, and provides a 
baseline against which the others are measured. For an alternative land use to be 
viable it must perform at least as well as the current land use. 

2. High-priority salinity prevention 

This option involves identifying saline and at-risk areas in the catchment, especially 
the discharge zones, and placing farm forestry on the associated recharge zones, 
higher in the catchment. 


1 Final definitions were the Transitional period, that time during which the new forestry industry 
is being established in the catchment, also defined as the Cost period, of the first five to 10 
years, and the Steady-state period or the Equilibrium period, the time when a forestry industry 
has been established in the catchment, at about year 30 to 40. However, only the results of the 
transitional time period are discussed in this paper. 

2 In this study the concordance threshold c* was set at 60% of the range and the discordance 
threshold d at 47% of the range. These high values ensured that any potential fatal flaws were 
reflected in poor performance of an option. 
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3. Medium-priority salinity plantings 

Similar to Option 2, but with a greater proportion of the recharge area planted. 

4. Additional under-used areas 

Similar to Options 2 and 3, but with trees planted on additional areas recognised as 
having limited value for conventional agricultural production, especially if adjacent 
to the priority salinity areas. 

5. Commercial plantations (with corporate land ownership) 

Medium-scale corporate investment in purchase of land and establishment of 
forestry in the higher rainfall areas of the catchment. 

6. Commercial plantations (with leased land) 

Similar to Option 5, except that land is leased by the corporation from landholders, 
potentially increasing the availability and decreasing the cost of land. 

7. Private medium-sized plantations 

Medium-scale forestry planting, undertaken by landholders. 

8. Agroforestry (plantations and grazing) 

Establishing of wide-spaced plantations, in conjunction with improved or native 
pasture or even fodder crop strips in more fertile areas. 

The Criteria for Evaluating the Farm Forestry Options 

The evaluation criteria have been defined considering the three major perspectives in 
land-use management, namely economic, environmental and social (including 
cultural) considerations. This approach is also useful in assessing the extent to which 
the forestry options achieve the goals of sustainable development, as defined by 
IDRC (1996). 

Table 1 lists the performance criteria used to evaluate the forestry options 
considered in the Hodgson creek MCA. The ‘rank’ columns in Table 1 indicate the 
relative importance of the criteria for evaluating farm forestry options. This relative 
importance was defined by landholders (broadacre farmers) in the lower section of 
the Hodgson Creek catchment. In this list, 1 denotes a criterion of high importance, 
through to 5 for least importance. 
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Table 1 . Rankings of the performance criteria for evaluating farm forestry 

Economic criteria Rank Environmental Rank Social criteria Rank 

criteria _ 

Shelter effects 2 


Forestry revenue - growth 
Forestry revenue - royalty 

Infrastructure costs 
(community) 

Regional impact 
Regional output 
Profit - farm 
Profit - regional 
Property value 

Rating treatment 

Risk profile 

Risk of policy change 

Equity of financial returns 

Cash flow - upfront costs 

Cash flow - debt 
servicing 
Critical mass 
Flexibility of land-use 
Liquidity of assets 


5 Soil resource 1 

quality 

4 Carbon 4 

sequestration 
Water quality 1 

4 Salinity control 1 

4 Biodiversity 2 

1 (local native) 

1 Water quantity 2 

3 Cumulative 2 

impacts 

4 Displacement of 3 

existing native 

2 bio-systems 

4 

4 Habitat quality 3 

2 Pest habitat 1 

2 Air quality 1 

(spraying of 
4 agricultural 

chemicals) 

3 
3 


Aesthetic amenity 1 

Change 4 

management 
requirements, 
including 
reskilling 


Consistency with 2 
local, state and 
federal 
government 
regulation and 
policy 


Net employment 3 

Maintaining 3 

services 

Community 3 

capacity 

Community 2 

cohesion 

Community 2 

acceptance 

Population turnover 2 

Equity 2 

Community health 2 

Health effects on 1 

farm families 


MCA EVALUATION OF FORESTRY OPTIONS FOR THE CASE-STUDY 
AREA 

Figure 2 presents in graphical form the aggregated performance of the farm forestry 
options. The panel on the left summarises performance according to the weighted 
summation aggregation technique, and that on the right according to Electre II. The 
upper bar charts presents the aggregate result of the environmental, economic and 
social criteria, while the lower sets of bar charts depict findings for the economic, 
environmental and social criteria individually. 
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Weighted Summation Electee II 



Figure 2. Outcomes of the MCA in Hodgson Creek 

In Figure 2, the left hand panel should be considered first. The height of each bar 
represents the performance of an option - the higher the bars the better the 
performance. The number that appears on top of each bar is the aggregate score for 
that option, i.e. the weighted sum of the individual scores against all the criteria. 
Considering the four bar charts in this panel, an option will only be considered to 
have high overall performance if it achieves average or high scores and no low 
scores in each bar chart, i.e. no poor performance in any of the perspectives 
(economic, environmental or social). The panel on the right of Figure 1 presents the 
aggregated score using the Electre II aggregation and should be considered second. 
These bar charts display the ordinal ranking of the options; the numbers that appear 
in these bar charts reflect these rankings. The highest performing option is ranked 1, 
second highest 2, and so on 

The weighted summation is used to assess the overall performance of the options 
and Electre II is used to test for the presence of fatal flaws. If the rank achieved by a 
particular option under Electre II does not reflect the rank it receives using the 
weighted summation, it is likely that there are one or more poor performing criteria. 
These criteria should be identified from the effects table. The analyst should then 
consider whether the poor performance in these criteria represents a fatal flaw in this 
option or if they are compensated for by high performance in the other criteria. 
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Comparison of Relative Performance of the Options According to Economic, 
Environmental and Social Criteria, Individually and in Combination 

Using economic criteria alone ‘Current practice’ is the best option, with high overall 
performance and no critically poor performance. Other options that perform well 
considering only economic considerations are the ‘High-priority salinity plantings’ 
and ‘Commercial plantations (leased land)’. The salinity plantings are small in area 
and in generally under-productive areas so do not displace other land uses, while 
‘Commercial plantations (leased land)’ pay annuities, thus increasing the 
performance in the profit and cash flow criteria. 

Using environmental criteria alone, the forestry options outperform ‘Current 
practice’. The large-area forestry options, including the ‘Agroforestry’ option, 
perform better than the small-area forestry options. 

Using social criteria alone the ‘High-priority salinity plantings’, the ‘Medium- 
priority salinity plantings’ and ‘Additional under-used areas’ perform well in both 
the weighted summation and Electee II analyses, with ‘Additional under-used areas’ 
performing the highest. 

Using all three major perspectives (economic, environmental and social) 
aggregated, under Weighted Summation ‘Current practice’ performs best, and 
‘High-priority salinity plantings’ and ‘Medium-priority salinity plantings’ perform 
well, followed by ‘Commercial plantations (leased land)’ and then the other options. 
There is a general trend that performance drops as the area trees planted increases. 

Using all three major perspectives, under the Electre II analysis, the results are quite 
different. The ‘Commercial plantations (leased land)’ option is ranked first, all the 
other forestry options second and ‘Current practice’ last. ‘Commercial plantations 
(leased land)’ perform well overall as the scores for the individual criteria are 
generally high and it has few exceptionally poor performing criteria. This option 
scores poorly in the economic criterion of liquidity of assets (as do all the forest 
options) but no others. The drop in performance of ‘Current practice’ is due to 
critically poor performance in a number of environmental and social criteria. 

Overall Conclusions for the Multi-Criteria Analysis 

All the forestry options outperform ‘Current practice’ from the environmental and 
social perspectives, but from an economic perspective ‘Current practice’ 
outperforms all of the forestry options. For forestry to be a viable option in this 
catchment, measures need to be taken to ameliorate this poor economic 
performance. The drop in performance is due to critically poor performance in the 
criteria habitat quality, shelter effects, soil resource quality, carbon sequestration, 
net employment, maintaining services and community capacity. These poor 
performing criteria, if present in an area of high concern, may prove to be fatal flaws 
in the management practice. Thus the technique of using compensatory and non¬ 
compensatory techniques - the weighted summation and Electre techniques - has 
shown high utility in assessing natural resource, environmental and land-use 
management practices in terms of overall performance and fatal flaws. 
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The option ‘Commercial plantations (with leased land)’ is most promising in this 
catchment followed by the option ‘Additional under-used areas’. The ‘Agroforestry’ 
option also merits further investigation. ‘Commercial plantations (with leased land)’ 
is considered the most promising option largely because it is the best performing 
forestry option from the economic perspective. This is largely due to the payment of 
annuities to the landholder by the organisation leasing their land. This option has 
high overall economic and environmental performance. Air quality was a major 
environmental consideration for landholders present at the stakeholder meeting. 
Large-scale forestry requires aerial spraying of agricultural chemicals, and 
alternative methods of applying these chemicals would need to be considered for 
this option to be acceptable in the Hodgson creek catchment. ‘Commercial 
plantations (with leased land)’ has average performance in the social perspective 
under the weighted summation analysis and poor performance in the Electee II 
analysis. This is largely due to a drop in performance in the criteria net employment 
and maintaining services. However, all the options perform poorly against this 
criterion, suggesting that the root of these problems lies elsewhere and will not be 
solved in the long term by introducing a forestry industry in the catchment. This 
option does however outperform ‘Current practice’ in both of these criteria. 

The option of forestry in ‘Additional under-used areas’ is worthy of further 
consideration, especially if the areas are also those identified where forests assist in 
salinity prevention. This option has high performance from the environmental and 
social perspectives. Environmentally, it does not perform as well as the larger area 
forestry options. The effect on the regional economy would be small because the 
planted area is small, and the option has low scores in critical mass and farm and 
regional profitability. This poor performance could be ameliorated by large-scale 
forestry being established in neighbouring areas. This small-scale forest industry 
could utilise the resources of a nearby large-scale plantation forestry industry, and 
therefore is worth considering in combination with the previous option. 

Whilst the ‘Agroforestry’ option has major weaknesses in key economic criteria, 
it has high performance from both the environmental and social perspectives. Also, 
there is anecdotal evidence that agroforestry will increase grazing productivity. Low 
scores in the important economic criterion of farm profit may however be 
insurmountable. 


DISCUSSION 

Multi-Criteria Analysis has proved useful for the integrated and holistic evaluation 
of projects and management options in natural resource and environmental 
management. This study demonstrates the utility of MCA for assessing farm forestry 
options on the Darling Downs from an economic, environmental and social 
perspective as well as from an overall perspective. Previous MCA studies have 
tended to lose detail of the performance of practices in particular areas of concern; 
overall good performance is assumed to compensate for poor performance in 
specific areas and the presence of fatal flaws has been overlooked. This does not 
reflect the concerns of the wider community and may result in poor decisions. The 
use of compensatory as well as non-compensatory multi-criteria analyses addresses 
this problem. The compensatory technique has provided with a sound measure of 
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overall performance of a management practice, whereas the non-compensatory 
technique alerts the decision-makers to presence of poor performing criteria. 
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